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ABSTRACT 

The 1972-73 accountability study of Duval County 
(Jacksonville, Florida) prograos for trainable aentally handicapped 
(TUB) children and youth, funded under Title VI-B of the Eleaentary 
and Secondary Education Act, focused on the developaent and 
refineaent of evaluation aaterials and field tisting of the oaterials 
developed. Twelve teachers identified approxlnt tely 550 behavioral 
objectives vith corresponding criterion test iteas and accoapanying 
instructional suggestions. Field testing involved evaluation of the 
550 behavioral objectives by teachers, parents, and coaaunity agency 
personnel, evaluation of student perforaance using criterion tes-L 
iteas, and evaluation of the project products and activities by a 
three aeaber panel of experts. Results of field testing supported 
such conclusions as the following; (1) that there is a high degree of 
correspondence between the iaportance attributed to skill areas by 
teachers, parents, and coamunity agency personnel with social skills 
generally ranked highest and occupational skills lowest; (2) that 
project aa^erials have a high probability of being used and 
correspond to teacher needs; {3| that the criterion test itea 
approach to assessaent of THH student perfortiance is inadequate; and 
(4) that there is need of additional behavioral objectives in the 
occupational skills area which take account of skills needed for 
success in sheltered workshop situations. (GV) 
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INTRODUCTION 



This report represents a sundry of the activities and results of the se- 
cond year uf the "Accountability Study of the Program for Trainable Mentally 
Retarded Children and Youth." This study is being conducted by the Duval 
county School Board. Jacksonville. Florida. Financial support was obtained 
under Title VI-B of the Elementary and Secondary Education Act. 

For infomation on the first year of this study, two documents. Suwnary 
evaluation Report (1971-1<>7?) and Accountability Mod.1 . should be referenced. 
These publications and additional Information on this study are available 
through the Clearinghouse/Infonna.ion Center, Bureau of Education for Ex- 
ceptional f tudents, 319 Knott Building, Tallahassee, Florida 32W4. 

Instructional Programing Associates, under the directidh of Dr. Keith Brown, 
has prepared tnis 19^2-73 sunr^ary report as a part of the evaluation activities 
of the second project year. 

The activities described in this report could not have been conducted without 
the sustained direction of Mr. William Geiger, the project director; excellent 
and extensive input from Dr. Charles Forgnone, Or. Wilson Guertin. Dr. Andrew 
Oseroff. and Dr. Louis Schwartz; and the support of the Exceptional St:rfent 
Education Section of the Duval County School Board i^nd the Bu«s«u of Educa- 
tion for Exceptional Students of the Florida SUte Dep>. tment of Education. 
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The contributions of the principal and teachers in the Duval County program 
for trainable mentally retarded (TMR) students cannot be underestimated. 
Their willingness to try new. untested evaluation procedures was the nc-l 
critical factor in making this project possible. 



BACKGRO'ifD 



"The Accountability Study for Trainable Mentally Retarded Children and Youth" 
was initially fiwded in July, 1971 with the anticipation of renewed support 
for a three year period. This project was originally submitted In response 
to a need recognired by the Bureau of Education for Exceptional Students of 
the Florida SUte Department of Education for a systematic means of evaluating 
the progress of trainable students in achieving functional skill levels as 
a result of the educational programs provided for them. Such information 
was perceived as being highly desirable for responsible adminifv.ative de- 
cision-making. The overall objective of the first project year was "to 
develop, implement, and audit an evaluation design based on systems analysis, 
which (would) measure the cost-effectiveness of the Duval County program for 
Trainable Mentally Retarded Children and Youth in achieving Its stated goals 
in the areas of language development, social adequacy, and vocational readiness". 
The results of this first year are contained in Summary Evaluation Report 
1971-1972 ■ 

As a consequence of the first year's successes and failures, several changes were 
made in the focus of the project. The major change was the development of a 
conceptual model for viewing a toUl program for TMR students. This model 
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possessed the necessary flexibility for easy use by teachers in many different 
settings and included an assessment technique which would; (!) provide 
teachers with useful information about student performance and (2) furnish 
a<toinistrative personnel with helpful program evali«tion data. 



PROJtCT PURPOSE AND OBJECTIVES 



The following were the objectives for the 1972-73 project year: 

Main Objecti ve: 

To modify, expand* and validate the evaluation of the cost- 
effectiveness of change in trainable mentally retarded stu- 
dents* behavior in all areas o^ concern necessary for ade- 
quate functioning in society, in such a way that the rnodel and 
most of the objectives {could) be utilized state-wide after 
the completion of the 1972-73 project year. 

Sub-objectives : 

* To revise the objectives developed during the first project 
year in the areas of vocational readiness, social adequacy^ 
^nd lanquage. to serve as the first set of objectives for 

a Si,dvr--w-*de catalog in the area of the trainable mentally 
retarded so thai ^hey meet 3*1 specifications of the state 
and can be utilizeo any program in the state. 

* To develop pre-objecti ves and objectives for the other areas 
of importance not Included in the original ^^^^ect, covering 
all areas of ir^^urtance for the trainable child from 6 to 18 
years of age. 

* To obtain validation of the objectives by teachers in similar 
programs across the state. 

* To validate the objectives by the opinion of parents. 

* To validate the objectives by the opinion of community agencies 
which provide services for the child before and after he leaves 
the educational program. 
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* To Incorporate into ti-.e f«)dified evaluation design all of the 
objectives. 

* To complete developront of methods, activities, and resources 
to be utilized in achieving a portion of the objectives. 

* To provide a comprehensive accountability report on the effect- 
iveness of resource utilization in changing student behavior. 

* To develop a training manual and a set r 30 tapes of be- 
haviors which require a high dev»^e of . judgement to de- 
termine if the behavior meets the criterl- in the objective. 



PROJECT ACTIVITIES 

As in any project of this scope, the activities carried out in the course 
of the year were diverse. The project activities focus sed on two major 
areas, (1) development and refineff«nt of materials and (2) field-testing 
of the materials developed. 

Development and ftefinement of Materials : 

A seven week workshop involving twelve teachers in the Duval County 
THR program was conducted in July and August, 1972. During this work- 
shop approximately 550 behavioral objectives with corresponding cri* 
terion test items and accompanying instructional suggestions were 
identified and written. The development and organization of these 
objectives into a flexible content- based model was guided by project 
consultants, Dr* Charles Forgnone* University of Florida, an< Drs. 
Andrew Useroff and Louis Schwartz, «^lorida State University. Editing 
of tne objectives and criterion test items and revision of the organ- 
izational model was conducted by the project staff through the Autumn 
0^ 1972, 

4 
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Thfc remainder of the project year was devoted to the refinen^nt and 
validation of the collection of 550 objectives and corresponding 
criterion test itero to determine their effectiveness as program 
evaluation Instruments. In tha refinement and validation of the 
1972-73 collection of objectives* the follMing activities were 
conducted: 

ft 

U The project staff contacted six counties having public 
school educational programs that served 150 or more 
trainable students to determine if personnel from these 
counties would assist in refining* ranking* and validating 
the objectives. The Florida counties which participated 
were; Dade* Broward* Palm Beach* Polk* Orange* and Hills- 
borough. A Saturday workshop was held in each county for. 
teachers and other county staff involved in the educational 
program for TMR students. During each workshop the purpose 
and history of the project and the organisational ir^del were 
presented. Each teacher then performed the following tasks: 
a. Rc:ici at least 10:; of tho objectives for clarity and 
indicated desired cha^geL. 

Ranked the importance of each oNiective* each specific 
skill area (module)* ani^ each general skill area 
(cluster)* on a five-point scale. 

c. Identified skills valuable fo'- TMR students that had 
been omitted from the collection. 

d. Responded to an anonymous opinion questionnaire about 
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the compr:;*ons4veness» value » and usefulness of the 
collection of objectives. 

2. At the beginning of the 1972-73 school year, parents of TW? 
stuiJents in the Duval County program were given a general 
orientation to the purpose and activities of the project 

at a PTA meeting held at Palm Avenue Exceptional Child Center. 
At subsequent PTA meetings, 97 parents conpleted an importance- 
ranking of the 26 clusters on a five-point scale* In addition, 
53 of these parents were contacted individually and cwipletcd 
a forced- ranking of the clusters using a Q-Sort technique. 

3. The Duval Association for Retarded Citizens, the Division 

for Mental Retardation's Regional Center, Vocational Rehabili- 
tation, United Cerebral Palsy, and Pine Castle School (sheltered 
workshop) were contacted during the Suftner of 1973. Thirty- 
four staff members from these organizations ranked the objectives, 
modules, and clusters for importance and indicateo skills that 
had been anitted from the catalog of objectives, 

Field-Testing of Materials : • 

Based on an evaluation design prepared by the project staff and Dr. 
Wilson H. Guertin, evaluation consultant to the project, the following 
activities were conducted to provide information on individual student 
performance and to determine the feasability of using criterion test 
items to evaluate the performance of TMR students: 
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K Ttw teachers of TWJ students in the Duval County program 
completed an informal assessment of each student's observed 
ability on the ccwiplete collection of objectives, lists of 
appropriate objectives for each student* compiled frora the 
informal assessment forms, were later given to each teacher. 

2. The ten mst important objectives for each student were ranked 
according to instructional priority for that «itudent* A de- 
scription of the top three to five objectives chosen for each 
student was sent hcm^ for the parents' information. 

3. The teachers administered criterion test items (pre-test) 
to each student. The test items corresponded to the top 
priority objectives which were selected as appropriate for 
the individual student, {The number of test items adminis- 
tered to each student by a teacher varied from one to about 
five.) 

4. To determine some interrelations an^ng items* the criterion 
test items for each objective in the Writing and Counting 
and Numeral Identification Clusters were administered to all 
of the students (252) in the program. 

OVERALL EVALUATION DESIGN 

The evaluation design for the second year of this project had three 
dis«.inct but inter-related aspects. These were: (1) evaluation of the 

7 
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materials by teachers, parents, and comminity agency personnel; (2) evalua- 
tion of student j^rformance using criterion test items; and (3) evaV»ation 
of project products and activities by a three-member panel of experts. 

Evaluation of the Collection of Objectives > Modules, and Clusters by 
Teachers »T^arents, and Community Agency Personnel : 

1. Evaluation by Teachers: 

In order to determine the comprehensiveriess* claf ity, and validity 
of the objectives and related materials, 122 teachers from seven 
Florida counties with prograr^ serving large numbers of TWl 
students: 



a. r^jad the objectives for clarity and indicated changes; 

b. ranked the importance of each objective, module, and cluster 
on the following five-point scale: 



c. reconsnended additional skills which they thought were 
valuable for trainable students but were not included in 
the collection of 550 objectives; and 

d. responded to an anonymous opinion questiomiaire about the 
comprenensiveness, value, and usefulness of the collection 
of objectives. 



2. Evaluation by Parents: 

To determine the comparative importance of various skill areas in a 
public school program for TMR students, the parents of trainable 
students in the Duval County Program were asked to complete a ranking 
of the 26 clusters which represented the collection of 550 objectives* 



essential 



2 3 

important not very 
important 



4 

unimportant 



very 
unimportant 
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Tte five-point scale described atyove was also used in this impor- 
tance-ranking activity. Ninety-seven parents ranked these clusters • 



These parents were also asked to take part in an individually ad- 
ministered, forced- ranking of the clusters using a Q-Sort technique. 
The distribution for this five-level Q-Sort was as follws: 



Most 


More 




Less 


least 


Important 


IfiiDOrtant 


Important 


Important 


Important 


2 


4 


14 


4 


2 



The fifty-eight parents who took part in this activity were asked to 
sort cards representiig the 26 clusters into the above distribution. 
Due to the len^thiness of ranking all of the objectives, parents were 
asked to rank only the clusters. This ranking provided an ove?'dV. 
view of skill area importance. 



3. Evaluation by Conmunity Agency Personnel: 

Staff inembers of Duval County community agencies serving TMR students 
(Duval Association for Retarded Citizens, the Division of Ment«il Re- 
tardation's Regional Center^ Vocational Rehabilitation^ United Cerebral 
Palsy* and Pine Castle School) were asked to rank the objectives, 
modules, and clusters using the five-point scale described in the 
section on ranking by teachers. They were also asked to indicate any 
skill areas omitted frofv the collection of objectives. 



Evaluation of Stu d ent Performance 

There were two aspects to the design for the evaluation of TMR students* 
performance on specific objectives. 



The first was the application of the sequence of assessment actlvf r:e$ 
depicted in the following TOoel which indicates the major components 
in the evaluation design: 



Informal Assessment! 



1 Assignment of Priorities 



I Selection of Ob 



If Objective 
Mastered 



Parent Review of 
Top Priority 
Objectives 



ective to Work on 



1 Criterion Pre-Testinq | 



Continuous Mon^t^^ing j 



ICriterion Post-Testing! 



If Objective 
Not rtestered 



The Intent of this model was to outline assessiaent procedures for 
evaluating student performance on individually selected objectives* 
The Instruments used for assessment were the criterion test items 
developed for each objective. The following is a rm>re detailed de- 
scription of this model: 

1. Informal Assessment: 

The first step in the evaluation of the trainable student's per* 
formance was an infcrmal assessment of the student's skills* 
Each teacher was given the collection of 550 objectives. In 
addition, he received a chCwknst with a short written description 
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of each of the objectives. (One checklist was provided for 
each student.) 8y completing this checklist on each student, 
the teacher indicated which of the objectives was appropriate 
for eacn of his students. 

2. Assignment of Priorities: 

Based on the information from the informal assessment and the 
progress that the student had made during the interim, the 
teacher assigned priorities to the objectives. The priority 
of d given objective for a student was a combination of per. 
ceived importance of skill acquisition for the student and 
the teacner's instructional sequence. Through the process of 
inforflid] assessment, all of the objectives which were in- 
appropriate for the student (at that particular time) were 
eliminated. In tne ass^jnment of priorities activity, those 
objectives which were appropriate for the student were placetU 
in a hierarchy so that they could be assessp'^ am worked on 
fln orderly ^'ashion. 

I. PiijBjr.;, Keview of Top Pnui .*v Objectives: 

The three to five objectives receiviny t*- nig^est priority were 
transferred to If Lier form and sent home to the parents of the 
student. Thi. provided a basis for school-home conmunication 
about specific skills in the student's instructional program. 

4. Formal Criterion Pre-Testing: 

From the ordered set of appropriate objectives, the teacher 
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selected the objective(s^ ::.3t he intended to work on first. 
After the selection was made* tne teacher obtamca tttc- cri- 
terion test item(s) and any necessary testing materials for 
that particular objecti ve(s). The test item was tnen admin- 
istered to the student in the manner specified in the test 
item instructions. After administration of the test item{s)^ 
the teacher completed the recording form and turned it in for 
analysis- • 

If the student mastered the gbjective(s) on the pre-test. the 
teacher selected the next object ive(s} and administered the 
criterion test item(s) for that objective(s). If the student 
had not mastered tne skill* the teacher began an instructional 
program focussed on assisting the student in acquiring the skill. 

Continuous Monitoring of -tudent Progress: 
At the completion of the first administration of tne criterion 
test itan{s). the teacner received a checklist ^rhich listed the 
same Dreakdown of the skills involved in mastering the objective 
as was contained in tne criterion test item- This checklist 
was for the te^cher*s use in continuously monitoring student 
progress toward mastery of the objective. Each time a student 
mastered a new part of an objective^ this progress could be 
recorded, providing the teacher with instructionally useful 
infortTiation on the student's performance. 

Fonral Criterion Post-Testing: 
,vi,>n the teacher thought that the student had mastered the ob- 

12 



jectlve, he again obtained the criteHon test item and materials. 
Ihe criterion test item was readministered to the stuoent» and 
the recording fonn was turned In for analysis. 

If the student had mastered the objective, the teacher selected 
another objective on which to pre-test the student. If the stu- 
dent had not mastered the objective, the teacher continued in* 
struction and post-tested again at a later time. 

Tne second aspect of the evaluation of the performance of TMR students 
was to attempt to define a sequence and to identify interrelationships 
between checklist elements within a criterion test item of an objective 
and between objectives in a cluster. To accomplish this two clusters 
were selected on which the entire population of TMR students in the 
Duval County prograw was assessed. The objectives selected for this 
extensive criterion test item administration were those in the Writing 
ar»d Counting and Numeral Identification clusters. 

Evaluation of Project P ^ nel^ of Experts 

All materials developed during the 1972-73 prcject year were examined 
by a three-member project evaluation panel: Dr. Harold W. Heller* 
University of Alabama; Dr. Oliver Kolstoe, University of Northern 
Colorado; and Dr. Bobby Palk* University of Alabara. After studying 
these materials for two days, the panel ret from May 29-June U 1973 
to determine whether or not the project had met its stated objectives 
and to recoimend future activities. 
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Evaluation of Objectives, Modules, and Clusters by Teachers, t'drent s. and 
C tmmunlty Agency Personne'l 

1. Evaluation by Teachers: 

Teachers of TMR Students from Dade, Duval, Broward, Palm Beach, 
Polk, Orange, and Hillsborough counties in Florida ranked the Im- 
portance of each objective, niodule, and cluster; 122 teachers par- 
ticipated in these rankings. Each of these teachers ranked the en- 
tire collection of 550 objectives on a five-point scale of importance. 
(1.0 on this scale corresponded to "essential".) Due to the large 
number of objectives, only the 50 objectives receiving the highest 
ranking are presented in Table 1. (The complete teacher rankings 
are available upon request.) 



Ranking of objectives In order of Importance by teachers (50 highest ranked 
objectives). 



Table I 



Item Description 



Mean 



Standard 
Deviation 



Washing face and hands 
Brushing teeth 
Urination (female) 
Urination (male) 
Wiping Mucous from face 
Defecation 

Wiping food from face 

Knows dangerous objects 

Knws objects harmful If Ingested 

Wiping food from hands 

Picks up, chews, swallows solid food 

Crosses intersection with light 

Menstrual care 

Crosse? Intersection without light 
£• lowing nose 



122 
121 
121 
122 
122 
122 
122 
120 
119 
122 
122 
120 
122 
119 
122 



1.04 
1.00 

i.n 
i.n 

1.12 
1.12 
1.15 
1.15 
1.15 
1.16 
1.16 
1.17 
1.17 
1.18 
1.18 



0,20 
0.31 
0.34 
0,34 
0,40 
0.35 
0.44 
0.40 
0.36 
0.43 
0.39 
0.40 
0.38 
0.41 
0.41 
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Standard 


Item Description 


N 


Mean 


Deviation 


ConpreHensi on of safety words 


122 


1.18 


P. 38 


Telling authority you are sick 


122 


1.20 


0.40 


Taking a shower 


122 


1.21 


0.43 


Knows objects to avoid ' 


120 


1.21 


0.45 


Knows objects hara^ul to eyes 


121 


1.21 


0.40 


Knows objects harmful to ean> * 


121 


1.22 


C.44 


K.nows objects harmful to nose 


121 


1.22 


0.42 


Drinking from glass 


121 


1.26 


0.53 


Initiates own natre and address 


122 


1.26 


0.49 


Names flashcard safety words 


122 


1.26 


0.56 


Initiates naroe\ address, telephone 


121 


1.27 


0.51 


Walks safely on shoulder of road 


117 


1.28 


0.49 


Uses knife, fck, spoon 


120 


1.29 


0.51 


Passes sharp objects 


119 


1.29 


0.45 


Verbally reports fire to person 


121 


1.30 


0.47 


Comprehension of public signs 


122 


1.30 


0.49 


Repeats name, address, telephone number 


121 


1.31 


0.6< 


Turns on, tests water in shower 


122 


1.32 


0.5?, 


Opens and closes doors 


119 


1.32 


0.50 


Uses electric appliance safely 


119 


1.33 


0.49 


Follows fire drill procedure 


121 


1.^3 


0.49 


Uses zipper 


122 


1.35 


0.48 


Combs hair 


120 


1.35 


0.53 


Uses buttons 


122 


1.36 


0.48 


Writes first and last names 


122 


1.37 


0.62 


Behaves safely on playground 


121 


1.38 


0.49 


Names flasncard public signs 


122 


1.38 


0.61 


Identities body parts 


122 


1.38 


0.52 


Initiates sentences on natne, address 


122 


1.39 


0.58 


Grasps, picks up object 


122 


1.39 


0.50 


Uses Snaps 


122 


1.41 


0.49 


Puts on shoes 


121 


1.42 


0.53 


Selects clean dinnerv.are 


121 


1.43 


0.53 


Climbs stairs 


121 


1.43 


0.53 


Puts on socks 


121 


1.44 


X 0.54 



After ranking each objective, the teachers wert requested to rank each module 
on the sante five-point scale. Table 2 summarizes the 50 highest ranked rrodules. 
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Table 2 



Ranking of modules in order of importance by teachers (5Q highest ranked 
modules) • 



iLtfn) iJcSCrtpteion 




Mean 


standard 
tteviation 


OatntrS 


i dS 


1 ♦U9 


U. 31 




1 


1 11 


n di 




1 L.J> 


1 17 


fi 


UacHac ikni4 Aii^^ ac coif 


} 


1 1 


v.*fU 




'\7*> 
tCC 


i • 1 H 


U« J/ 




\ 9^ 
fC^ 


1 1 «i 
I . J D 




doff 1 &.auiun 




7.1/ 








1 1 Q 


A ^ A 

0-4U 




1 9> 


1 lO 
1*17 


ft 


Lav> >u 1 1 Q TUOQ 


1 C J 


1 


U.45> 




t ^ J 


1 'i'i 
1 * J J 


A /I Q 

U#4*7 




1 ^'5 


f « 


ft CO 






1 'J'J 
1 • J J 






*CI 


1 17 


A C 7 


use oT Uwensi is 


> 'ii 
iZl 


f • 3o 


A C ^ 

0.53 


ra>4.eniny ciOLnes 


t 91 
iCt 


1 .hU 


A C7 
0« DC 


UQpr vonvroi 


icl 


1 .40 


A no 


[>l/U^ par v3 


1 ^ 


t AC 


A C7 




i cS 


1 

1 • 


A CO 

0.59 


DvUjT tittayt? 




1 

1 « t>4 


ft CC 


l/^ciijs^ iw^^'AiF* { n 4 f^fi 


1 70 


1 


A 


n{ f^/"t*{ nnal 4 an/4 4 

Ui 1 1? V 4.1 una n v/ anu latcrollwy 


1 77 


1 . DO 


A C7 


OA/*^nn{ VAC r>i>'i nc 


1 7f1 


1 C7 


A CI 


r 1 r» w SI u 


1 77 


f CI 
1 «Dl 


A CC 


SKin ware 


1 77 


1 *ol 


A CC 




1 77 


1 Al 
1 •Ol 


ft C7 


Sookpn oroduction of 




1 « Q 1 


n Rfi 


Counting objects 


122 


1.51 


0.57 


Auditory discrimination 


122 


1.62 


0.59 


Balance 


121 


1.64 


0.59 


Visual discrimination 


122 


1.65 


0.63 


Opening containers 


122 


1.66 


0.54 


Recognizes paper nwey 


120 


1.67 


0.72 


Shop safety rules 


121 


1.68 


0.72 


Auditory association 


122 


1.70 


0.58 


Care of apparel 


122 


1.72 


0.55 


Dishwashing and drying 


122 


1.72 


0.69 


Spoken production of phrases 


122 


1.73 


0.63 


Tells time 


121 


1.73 


0.68 


Bedmaking 


121 


1.73 


0.64 


Auditory figure-ground 


121 


1.74 


0.61 
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It em Oescription 



N 



t4ean 



Standard 
Deviation 



Balanced eating and drinking 
Tidying 

Cleaning bathroom 
Clearing and setting table 
Hair dressing 

Comprehension of Sight vocabulary 
Food storage and retrieval 
Cleans kitchen 
Appropriateness of dress 



123 
122 
122 
121 
120 
121 
121 
122 
121 



1.75 
1.75 
1.75 
1.76 
1.77 
1.77 
1.77 
1.78 
1.80 



0.63 
0.55 
0.58 
0.62 
0.65 
0.68 
0.57 
0.66 
0.64 



Finally, the teachers were asked to rank the 25 clusters representing Social 
Skills, Basic Skills, and Occupational Skills. The results are presented in 
Table 3. It is interesting to note that all of the clusters representing 
the Social Skills were ranked in the top third; clusters representing Occu- 
pational Skills were generally ranked low. 

Table 3 

Ranking of clusters in order of importance by teachers. 



Item Description 



H 



Mean 



Standard 
Deviation 



Clean! ir.>ss 
Safety Skills 
Eating 

Health Skills 

Dressing and Undressing 

Listening 

Speaking 

Groomi ng 

Sensory Motor Coordination 
Position of Body in Space 
I'ouseke.fjing 
i.jmemaking 
Money Concepts 

Counting and Numeral Identification 
Maintenance 

Recreation Activities - Individual 
Recreation Activities - Group 



122 
122 
122 
122 
122 
122 
122 
122 
120 

m 

122 
120 
122 
122 
122 
121 
121 



1.06 
1.05 
1.11 
1.13 
1.24 
1.31 
1.39 
1.40 
1.51 
1.52 
1.63 
1.66 
1.67 
1.73 
1.73 
1.82 
1.90 



0.27 
0.27 
C.34 
0.34 
0.46 
0.46 
0.51 
0.49 
0.62 
0.60 
0.59 
0.56 
0.68 
0.65 
0.60 
0.69 
0.80 
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Item Description 








N 


Mean 


Deviation 


Writing 


122 


2.02 


0.81 


Use of Shop Tools 


122 


2.02 


0.83 


Measurement 


122 


2. in 


0.77 


Reading 


122 


- - 1 


0.88 


Recreationdl Arts 


122 


2.13 


0.85 


Personal Public Service Skills 


122 


2.17 


0.86 


Craft ^kIIU 


V. 


2.24 


0.86 


Addifiiii ant! "• jbtraction 




2.25 


0.91 


Cleritdl Service Skills 


?22 


2.40 


O.Pl 



In additiOT^ to ranking the obj^.^ives* iwdules^ and clusters, the 
teachers indicated poorly wordcj statements and possible additional 
objectives. Tni? Information v s usee 'n mnjifying individual ot>* 
jectivf- jnd in expanding the at ire co Hoc t* on • 



In order to determine Initial reaction to the project and its probable 
use by teachers across the state, the teachers from these six counties 
were asked to complete an anonymous opinion questionnaire. The response 
to this questionnaire was a positive endorsement of the potential use- 
fulness of Che materials that were developed by the project during the 
second year. 

2. Evaluation by Parents: 

Parents of students in the Duval County TMR program participated 
in two different cluster ranking activities. Ninety-seven parents 
completed a ranking of the clusters using the same five-point 
scale as the teachers. The rankings were completed during large PTA 
meetings. All of the items were read verbally to assist parents who 
had difficulty reading. The results of the parent group-ranking are 
presented in Table 4. 
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T4b1e 4 



Group- ranking of clusters by parents \n o^-der of importance 

. ~ ""^ Standard 
Item Description N Mean Deviation 

Cleanliness 97 ^,29 0.59 

Safety Skills 9$ ^.33 o.62 

9S 1.42 0.63 

'-^s^n^ig 9S 1.51 0.74 

Health Skills 97 1.53 5:63 

Speaking 9$ 1.56 o.72 

Dressing and Undressing 95 ) 5] g 62 

Grooming 94 t]^ 

Sensory Motor Coordination 93 1 6f 0 67 

Housekeeping 97 -^'y.j Q'gg 

Recreation Activities - Individual 92 t,'76 o'sO 

M^J^nS 96 1.79 0.'63 

Horemaking gg ,83 

Jteading 97 ^ Q4 

Position of Body in Space 92 1.84 

Counting and Numeral Identification 96 1*88 

Honey Concepts gg ^'gg „ 

Addition and Subtraction 93 2 00 0 72 

Jfaintenance % 2! 02 o;74 

^'^I^*^^??!^ '^'■^^ 94 2.04 0.70 

Slfr^iill^ 95 2.08 0.63 

Measurement 95 2 21 0 75 

Personal Public Service Skills 94 2*29 0*83 

"^''<"J service Skills 94 Z.'ss Q.n 

Use of Shop Tools 92 ? 33 n 05 

Recreational Activities - group 77 i'87 



0.67 
0.64 
0.74 
0.62 
0.68 



These parents were also asked to take part in an individually adminis- 
tered ranking of the clusters using a Q-Sort technique with a five- 
level distribution. Fifty-eight parents volunteered to cwnplete this 
ranking. The results of this parent Q-Sort ranking are presented in 
Table 5. 
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Table 



Q^Sort ranking of clusters by parents In order of importance. 



Item Description 

Speaking 

Listening 

Safety Skills 

Cleanliness 

Reading 

Health Skills 

Eating 

Dressing and Undressing 
Sensory Motor Coordination 
Writing 
Grooming 

Counting and Numeral Identification 

Housekeeping 

Money Concepts 

Recreation Activities - Individual 
Honiemaking 

Position of Body in Space 

Recreation Activities - Group 

Addition and Subtraction 

Craft Skills 

Recreational Arts 

Personal Public Service Skills 

Use of Shop Tools 

Measurement 

Maintenance 

Clerical Service Skills 



N 


lie an 


Standard 
Deviation 


S8 


1.93 


0.72 


58 


1.98 


0.84 


58 


2.Z8 


0.83 


58 


2.34 


0.76 


58 


2.43 


0.95 


58 


2.50 


0.73 


58 


2.57 


0.98 


58 


2.60 


0.76 


58 


2.64 


0.98 


58 


2.72 


0.78 


58 


2.81 


0.66 


58 


2.97 


Q.64 


58 


3.00 


0.56 


58 


3.12 


0.77 


58 


3.19 


0.66 


58 


3.19 


0.54 


58 


3.24 


0.77 


58 


3.24 


0.68 


58 


3.26 


0.88 


58 


3.41 


0.74 


58 


3.53 


0.68 


58 


3.69 


0.72 


58 


3.76 


0.88 


58 


3.78 


0.77 


58 


3.78 


0.87 


58 


4.03 


0.83 



Some Interesting differences exist between the two methods of ranking. 
With the 0-Sort technique, the relative position of several clusters 
changes; for example, listening, speaking, and reading skills move up 
in Importance. Clusters In the Social Skills make up the majority of 
the top third; clusters from the Occupational Skills make up the 
majority of the bottwt third. This Is similar to the results of the 
teacher ranking reported in Table 3. 
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i. Evaluation by Conmunity Agency Personnel: 

The staff of five cwrounity agencies providing services to 
students In Duval County were asked to rank the objectives, modules, 
and clusters. Thirty-four staff menters from the Duval Association 
for Itetarded Citizens, the Division of Retardation's Regional Center, 
Vocational Rehabilitation, United Cerebral Palsy, and Pine Castle 
School completed the rankings and reviewed the entire collection 
of objectives. Table S presents the top 50 objectives as ranked by 
these connunity agency personnel serving TMR individuals in Duval 
County. (The complete comnunity agency personnel rankings are 
available upon request.) 



Ranking of objectives in order of importance by community agency personnel 
(50 highest ranked objectives). 



Table 6 



Item Description 



N 



Mean 



standard 
Deviation 



Picks up, chews, swallows solid food 

Taking a shower 

Brushing teeth 

Washing face and hands 

Wiping mucous ftxm face 

Menstrual care 

Defecation 

Knows dangerous objects 

Wiping food from face 

Drinking from glass 

Wiping food from hands 

Knows objects harmful If ingested 

Turning on, testing water in shower 

Urination (female) 

Urination (male) 

Uses knife, fork, spoon 

Knows objects to avoid 

Grasps, picks up object 

Knows objects harmful to eyes 

Knows objects harmful to ears 

Kno»$ objects harmful to nose 



34 
34 
34 
34 
34 
33 
33 
34 
33 
34 
33 
33 
34 
34 
34 
34 
34 
34 
34 
34 
34 



1.26 
1.35 
J. 41 
1.41 
1.44 
1.45 
1.45 
1.47 
1.48 
l.W) 
1.52 
1.52 
1.53 
1.53 
1.53 
1.56 
1.59 
1.62 
1.65 
1.65 
1.65 



U.44 
0.48 
0.55 
0.49 
0.60 
0.50 
0.61 
0.78 
0.66 
0.56 
0-70 
0.61 
0.61 
0.81 
0.81 
0.60 
0.77 
0.54 
0.64 
0.64 
0.64 
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Standard 


Item Description 


N 


Mean 


Deviation 


Takes off clothes over shoulders 


34 


1.6t> 


0.76 


Initiates own name and address 


34 


1.65 


1.00 


Comprehension of safety words 


34 


1.65 


0.54 


Repeats name, address, telephone 


33 


1.67 


1.03 


Takes off clothes over head 


34 


1.68 


0.79 


Pulls down clothes 


34 


1.6P 


0.79 


Pulls up clothes 


34 


1.68 


0.79 


Puts clothes on over head 


34 


1.68 


0.79 


Puts clothes on over shoulders 


34 


1.68 


0.79 


Comprehension of public signs 


34 


1.68 


0.58 


Passes sharp objects 


33 


1.70 


0.76 


Shampooing hair 


34 


1.71 


0.71 


Crosses intersection with light 


34 


1.71 


1.02 


Takes off shoes 


34 


1.71 


0.82 


Takes off socks 


34 


1.71 


0.82 


Puts on socks 


34 


1.71 


0.82 


Puts on shoes 


34 


1.71 


0.82 


Uses buttons 


34 


1.71 


0.75 


Writes first and last names 


34 


1.71 


0.67 


Names flashcard safety words 


34 


1.71 


0.57 


Identifies coins 


34 


1.71 


0.67 


Names public signs 


32 


1.72 


0.62 


Opening and closing doors 


34 


1.74 


0.82 


Uses zipper 


34 


1.74 


0.74 


Blowing nose 


34 


1.74 


0.92 


Telling authority you are sick 


34 


1.74 


0.74 


Initiates name, address, telephone 


34 


1.74 


1.01 


Identifies bills to $20 


34 


1.75 


0.73 


Identifies bociy parts 


34 


1.76 


0.73 



Table 7 summarizes the 50 nwdules attributed the roost in^wrtance by 
34 Duval County conmunity agency personnel. 



Table 7 



Ranking of modules in order of importance by conmunity agency personnel (50 
highest ranked modules). 

=^="=^="= Standard 
Item Description N Mean Deviation 

Drinks liquids 34 1.15 0.35 

Bathes 33 1.21 0.41 
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Item tescrlption 

tats solid food 

Putting on clothes 

Washes and dries self 

Taking off clothes 

Uses utensils 

Fastening clothes 

Uses dangerous objects safely 

Wipes face 

Street safety 

Fire safety 

Sanitation 

Eye, ear* nose, and throat safety 

Recognizes coins 

Odor control 

Recognizes paper money 

Balanced eating and drinking 

Recreational safety 

Sorts, washes, and dries clothes 

Counting objects 

Bedmaking 

Weather safety 

Opening containers 

Sickness 

Tells time 

•Identification of currency by cent value 
Dishwashing and drying 
Exchanges money for up to 50t 
Balance 

Cleans kitchen 

Using kitchen appliances 

Spoken reproduction of words 

Spoken production of words 

Shopping 

Oral Counting 

Numeral identification 

Visual motor coordination 

Identifies price tags 

Makes change for up to Si. 00 

Locks 3nd unlocks 

Using stove 

Care of apparel • 
Skin care 

Clearing and setting ta&le 
First aid 

Auditory discrimination 

Addition 

Body image 

Body parts ^ 
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Standard 


H 


Mean 


Deviation 


jH 


1 .21 


0.40 


1/1 


1.29 


0.46 


34 


1.32 


0.47 


34 


1.32 


0.47 


33 


1, 39 


0.49 


33 


1 .39 


0.55 


33 


1 .42 


0.60 


33 


1.45 


0.86 


33 


1 .4o 


0,56 


OH 


T A'9 

1 .47 


0.61 


33 


1,48 


0.66 


31 


^ CD 

1 .5o 


0.61 


3£ 


1.72 


0.57 


33 


1.73 


0.66 


32 


1 .75 


0.61 


34 


1.79 


0.72 


34 


1.79 


0.63 


30 


1.80 


0.54 


34 


1.81 


0.73 


30 


1.83 


0.58 


3Z 


Y Oil 

i»84 


0.71 


30 


1 ^87 


0.67 


33 


l.p] 


0.69 


32 


l.bd 


0.70 


31 


1 .90 


0.64 


3U 


1 .90 


0.65 


32 


1.91 


0.63 


33 


i.9i 


0.75 


30 


1»93 


0.68 


30 


1.93 


0.68 


33 


1.94 


0.85 


33 


1 nA 

1.94 


0.81 


I** 
oc 


1 .94 


0.70 


3c 


1.94 


0.79 


32 


1.94 


0.75 


33 


1 .94 


0.69 


5c 


1 .97 


0.64 


32 


1 . 9/ 


0.73 


3! 


1.97 


0.65 






ft cc 




2.00 


0.55 


33 


2.00 


0.78 


30 


2.00 


0.58 


33 


2.03 


0.76 


33 


2.03 


0.76 


32 


2.03 


0.77 


32 


2.03 


0.64 


32 


2.03 


0.77 



ERIC 



The ranking of the clusters In order of ifi^ortance by coimtunity agency 
p "^onnel is presented in Table 8. The distribution of community agenQ' 
personnel -ranked clusters is very similar to the distribution of clusters 
as ranked by both teachers and parents. The top third enqjhasizes Social 
Skills; the bottom third emphasizes Occupational Skills. 

Table 8 

Ranking of Clusters in order of inf ortance by community agency staff. 



Item Description 



Eating 
Cleanliness 

Dressing and Undressing 

Safety Skills 

Listening 

Grooming 

Health Skills 

Speaking 

Sensory Motor Coordination 

Money Concepts 

Writing 

Housekeeping 

Coi^ting and Numeral Identification 
Maintenance 

Position of Bodty in Space 

Homemaking 

Reading 

Addition and Subtraction 

Recreation Activities - Individual 

Recreation Activities - Group 

Measurement 

Use -of Shop Tools 

Recreational Arts 

Personal Public Service Skills 

Clerical Service Skills 

Craft Skills 



H 


Mean 


Standard 
Deviation 


34 


1.15 


0.35 


34 


1.21 


0.40 


34 


1.32 


0.53 


34 


1.41 


0.55 


34 


1.47 


0.55 


34 


1.50 


0.56 


33 


1.55 


0.66 


34 


1.74 


0.66 


33 


1.88 


0.69 


33 


1.91 


0.67 


33 


1.94 


0.78 


33 


1.97 


0.67 


32 


2.00 


0.75 


32 


2.03 


0.64 


33 


2.12 


0.81 


32 


2.16 


0.67 


33 


2.21 


0.77 


32 


2.22 


0.65 


34 


2.24 


0.73 


33 


2.27 


0.75 


34 


2.41 


0.69 


33 


2.70 


0.80 


34 


2.74 


0.85 


34 


2.85 


0.97 


33 


2.88 


0.81 


32 


2.88 


0.82 



24 



ERIC 



Evaluation of Student Performance : 

Several problems were encountered In evaluating student performance. Two 
departures from the proposed design (page 10) were necessary. The con- 
tinuous monitoring and criterion post-testing aspects were not systemati- 
cally undertaken. This was due primarily to scheduling delays which re- 
sulted ih the initial testing being spread out over the entire year* 
However, the data collected and procedures used did result in useful 
modifications to the assessment procedures employed in the 1973-74 project 
year. The information collected during the 1972-73 field-test did little 
to clarify the level of performance of the stude««LS or the inter-relation- 
ships between objectives* and is therefo**-. lui presented in this report. 

The Duval County teachers who participated in the collection of pupil 
performance data were asked to coi^lete an anonynwus questionnaire 
to evaluate the value of the 1972-73 assessment activities. They were 
provided the following written instructions prior to completing the 
questionnaire: 

INSTRUCTIONS: This questionnaire is being given to you in order to 
obtain your feelings about your participation in the 
field-test activities of the TMR Accountability Study. 
This information is sought in order to improve the 
study next year. Your frank answers will be greatly 
appreciated. Your response should reflect the way 
you think or feel about each statement, in terms of 
the following six-point scale. 

STRONGLY MILDLY MILDLY STRONGLY 

DISAGREE DISAGftEE DISAGREE AGREE AGREE AGREE 

SD 0 MO MA A SA 

1 2 3 4 5 6 
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Tte questions ant' resuUs are presented In Table 9. These results sup- 
ported possible benefits from participating in the study. However, scmie 
detailed changes were indicated, such as, movement away from criterion 
test item assessment procedures. 



Table 9 



Reactio'- of Duval County Teachers to par'** pc^ion in the 1972-73 field-te^t 
activities of THR Accountability Studv. 



^ St atement N Mean 

My 1nvolvef!ient in the Accountabllitv Study 
ha- made f» better program for cH Idren. 

The criterion reference t^rsting tak^s more 
time than it is wortb. 

I received adequate training in order to 
be able to assess ay students* skills. 

The criterion reference testing provided 
useful information in determining exactly 
how the students were doing in a skill. 

The informal assessment {i*e*, completing 
the checklist of objectives) on each 
student was a useful activity. 

The Saturday workshops were a waste of 
tiTOp 

I knew how students would do on the 
criterion reference tests before I 
administered them. 

The Accountability Study has the potential 
of providing something useful to TMR 
teachers. 

The information from the informal assess- 
ment was used in planning instruction for 
my students. 



21 3.28 

21 4.28 

21 4.81 

21 3.28 

21 3,28 

21 3.33 

21 4.52 

21 4.28 

21 3.43 
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Statenent 



N 



Mean 



If I had a choice T would use crIceHon 
refetence tests to evaluate rqy students. 



20 



3.30 



If participation in the Tf« Study were 
voluntary I would not parti cif»te. 



21 



I could have accurately assessed students 
performance on the objectives without 
administering the criterion reference tests. 



20 



4.35 



The Informal assessment gave a useful pro- 
file for planning an instructional program 
for a student. 
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3.66 



I could have provided a tetter program for 
students If I did not have to spend time 
on the AccountabllltyTtudiy. 21 3.66 



Evaluation of Project by Panel of Experts : 

The fbllwing represents a sunmary of the report of the three-member 
evaluation panel. 



* 90? of the objectives met the minimum criteria for 
(^rformance objectives adopted by the State of Florida. 

* At least 90* nf the items included in the 6(X) plus 
objectives are Important to the development of the 
youngsters. 

* The present voliAne should edited to reduce each 
behavioral objective to the least possible number of 
words without losing the purpose* clarity, ^nd meaning. 

* Visits should be macte to approximately 5 sheltered 
workshops to Identify appropriate objectives for the 
16 to 18 year old TNI students. 

* A users* manual should be developed to accompany th€ 
collection of objectives. 

* The methods and materials aspect of the model should 
be bound in a separate book as a resource for the 
collection of objectives. 
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COfWILUSIOMS AND IMPLICATIONS 

The results of the second year of the project leatf to the follwing con- 
clusions: 

1. There Is a high degree cf corresponderrce between the importance 
attributed to skill areas by teachers, parents, and ctHnmunlty 
agency personnel with Social Skills ranking generally at the 
top and Occupational Skills ranking at the bottom. 

2. The materials developed during the 1972-73 project year have a 
high probability of being used and fill a need in the field 
based upon teacher review of the material and review by the 
panel of experts. 

3. The criterion test item approach to assessrent of TMR student 
performance is inadequate based upon feedback from teachers 
using the technique. Less reliance on structured criterion 
test items seems necessary If systematic evaluation of student 
performance is to be adopted by teachers. 

4. Expansion of the number of objectives in the Occupational Skills 
area Is necessary and should take into account those skills needed 
for successful performance in sheltered workshop situations. 
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